Method for recognizing activated lamps at a vehicle

ABSTRACT

A method for recognizing which lamps at a vehicle are activated. The method includes: providing multiple image recordings of candidate areas at the vehicle in which an activated lamp is presumed; converting the image recordings into an intermediate product by executing a recurrent encoder network (ERNN), the output of at least one pass of the ERNN is supplied as input to a further pass of the ERNN, and different image recordings of candidate areas are supplied as input to different passes of the ERNN; assignments of the image recordings of candidate areas are ascertained to classes which represent specific lamps of the vehicle from the intermediate product by executing a recurrent decoder network (DRNN) multiple times, the output of at least one pass of the DRNN is supplied as input to a further pass of the DRNN.

CROSS REFERENCE

The present application claims the benefit under 35 U.S.C. § 119 ofGerman Patent Application No. DE 10 2022 200 136.2 filed on Jan. 10,2022, which is expressly incorporated herein by reference in itsentirety.

FIELD

The present invention relates to the recognition of which lamps at avehicle are activated, in particular, for planning the behavior of afollowing ego-vehicle.

BACKGROUND INFORMATION

In road traffic, it has been customary for quite some time for vehiclesto be equipped with brake lights and turn signals. With these lamps, thedriver of the vehicle is able to signal to other road users whichactions he or she is planning in the near future. It is thereforedesirable to also take these useful pieces of information intoconsideration in driver assistance systems and systems for the at leastsemi-automated driving, for the planning of future interventions in thedriving dynamics of the vehicle.

An exemplary method for recognizing the state of lamps at a vehicle isdescribed in China Patent Application No. CN 111 881 739 A.

SUMMARY

Within the scope of the present invention, a method for recognizingwhich lamps at a vehicle are activated is provided. The vehicle may, inparticular, be a vehicle preceding an ego-vehicle, for example, and themethod may then be carried out on board the ego-vehicle.

Within the scope of the method, multiple image recordings of candidateareas at the vehicle are provided, in which an activated lamp ispresumed. These image recordings may, in particular, be partial images,for example, which were previously selected from a larger still image orvideo image.

According to an example embodiment of the present invention, the imagerecordings are converted into an intermediate product by executing arecurrent encoder network (ERNN) multiple times. In the process, theoutput of at least one pass of the ERNN is supplied as input again to afurther pass. In addition, different image recordings of candidate areasare supplied as input to different passes of the ERNN. The ERNN canthus, for example, be supplied a first image recording as input duringits first pass. The second pass may then, for example, receive theoutput of the first pass, and additionally a second image recording, asinput. The third pass may then, for example, receive the output of thesecond pass, and additionally a third image recording, as input, and soforth.

From the intermediate product, which is obtained after all passes of theERNN have been completed, assignments of the image recordings ofcandidate areas to classes are subsequently ascertained by carrying outa recurrent decoder network (DRNN) multiple times. These classesrepresent specific lamps of the vehicle, such as, for example, rearlights, brake lights, turn signals or rear fog lights. Additionally, itis possible, in particular, for at least one class to be provided, forexample, for areas for which the presumption that an activated lamp ofthe vehicle is present therein proves to be incorrect. Examples of suchareas are reflections of foreign light at the vehicle, in particular,for example, at the license plate, which is specifically designed to bereflective.

According to an example embodiment of the present invention, in theprocess, the output of at least one pass of the DRNN is supplied asinput to a further pass of the DRNN. (See, FIG. 1, 131 ). In each passof the DRNN, an assignment of the image recording which was processed inthis pass of the ERNN to at least one class is ascertained. (See, e.g.,FIG. 1, 132 ). In the first pass of the DRNN, the assignment of theimage recording which was supplied to the first pass of the ERNN asinput to a class is ascertained. In the second pass of the DRNN, theassignment of the image recording which was supplied to the second passof the ERNN as input to a class is ascertained, and so forth.

It was recognized that, through the use of a recurrent ERNN and arecurrent DRNN, the task of recognizing all activated lamps which maypossibly be present in a still image or a video image of the vehicle maybe broken down into a plurality of subtasks. These subtasks may each becompleted by one pass of an ERNN or a DRNN, which includes only a smallnumber of neurons. Such an ERNN or such a DRNN may also be executed onhardware which does not provide a particularly powerful CPU and also nospecial hardware accelerators for the execution of neural networks. Incontrol units for driver assistance systems or systems for the at leastsemi-automated driving of vehicles, for example, the electrical powerconsumption and/or the possible heat dissipation is/are limited.

At the same time, the individual image recordings of candidate areas, asa result of the recurrent execution of the ERNN and the DRNN, are notconsidered in an isolated manner, but are in each case considered in thecontext of the remaining image recordings. Both when creating theintermediate product and when ascertaining classes from the intermediateproduct, results which were obtained during the previously carried-outprocessing with respect to other image recordings are also always reusedduring the processing with respect to each image recording. By alsoincorporating the context in the surroundings of the vehicle lamp,activated lamps may be classified particularly reliably, and this mayeven be achieved from a single image. The recognition of activated lampsusing traditional image processing, in contrast, frequently requiresmultiple chronologically consecutive frames and is accordingly slower.

In one particularly advantageous embodiment of the present invention,the candidate areas are selected based on their luminance and/or colorfrom an image or a video of the front or the rear of the vehicle. Forexample, such candidate areas may be ascertained and/or tracked bythreshold values in luminance and/or color. In this way, all candidateareas for which there is at least an “initial suspicion” that they showactivated vehicle lamps may be ascertained very quickly. All thesecandidate areas may then be sequentially converted into the intermediateproduct with the aid of the ERNN, and subsequently be analyzed with theaid of the DRNN. Regardless of how many candidate areas there are, thesame hardware complexity is always required, just as a supermarket, inprinciple, also remains functional with only one open cash register,regardless of the number of customers.

According to an example embodiment of the present invention, the imagerecordings of the candidate areas may be supplied to the ERNN, forexample, directly in their raw form as arrays or tensors includingintensity and/or color values of pixels. In one particularlyadvantageous embodiment, the image recordings, however, are condensedinto feature vectors which indicate the peculiarity of one or multiplefeature(s). These feature vectors are even supplied to the ERNN asinput, instead of the image recordings. In this way, the dimensionalityof this input may be reduced and, at the same time, also bestandardized. Just the standardization is advantageous, since thecandidate areas may have very different sizes. For example, an areawhich includes a rear light and a turn signal, situated next to it, ofthe vehicle may be considerably larger than an area which randomlyreflects the incident light from the street lights in the direction ofthe camera.

In one further advantageous embodiment of the present invention,candidate areas are selected with the aid of a further neural networkfrom an image or a video of the front or the rear of the vehicle. Such afurther neural network may combine the ascertainment of the candidateareas and the extraction of features from these candidate areas in oneoperation. The further neural network may, in particular, include atleast one convolutional layer, for example, which, by applying at leastone predefined filter kernel to its input, ascertains a feature map ofthis input. This feature map may then be supplied to the ERNN as input,instead of the image recordings.

In one further advantageous embodiment of the present invention,positions and/or sizes of these image recordings relative to the vehicleare supplied to the ERNN as input, in addition to the image recordingsor feature vectors or feature maps formed thereof. This additionalcontext information may then also be converted by the ERNN into theintermediate product, and subsequently be evaluated by the DRNN. In thisway, it is possible, for example, for learned knowledge to be taken intoconsideration as to the locations at the vehicle at which the occurrenceof certain activated lamps may be plausible at all. Turn signals aresituated at the two sides of the vehicle, for example, but not in thecenter.

In one particularly advantageous embodiment of the present invention, anERNN and/or a DRNN having no more than 100 neurons is/are selected. Anetwork of this size may also be executed on hardware which does nothave any resources specifically designed for the execution of neuralnetworks, such as, for example, GPUs, particularly rapid CPUs or aparticularly large working memory. At the same time, the network,however, is still flexible enough to be able to manage the particularsubtask well, namely the processing of pieces of information from anindividual candidate area or the classification of such a candidatearea.

Particularly advantageously, according to an example embodiment of thepresent invention, the image recordings are recorded with the aid of atleast one sensor which is carried along by an ego-vehicle. Theego-vehicle is a vehicle whose optimal driving behavior in the moreimmediate future may depend on which lamps at the vehicle analyzed withthe aid of the method are activated. The analyzed vehicle may, forexample, be a preceding vehicle or also an oncoming vehicle.

If the brake light is activated, for example, in a preceding vehicle,this indicates that the preceding vehicle is decelerating. Theego-vehicle may then also gently decelerate in time, even before itsdistance with respect to the preceding vehicle drops below a criticalvalue, and a sudden stronger brake application becomes necessary.

If the turn signal is activated in the preceding vehicle, this showsthat the preceding vehicle intends to change lanes or to change thedriving direction. When the preceding vehicle intends, for example, toturn right at a right angle, the ego-vehicle has to prepare for the factthat the preceding vehicle will become considerably slower prior to theturning process. The ego-vehicle on its part also has to prepare forstopping behind the preceding vehicle, when the preceding vehicle, onits part, is required to wait compared to other road users, such aspedestrians or bicyclists who want to continue straight ahead. In thisregard, starting with the recognition that the preceding vehicle intendsto turn, a behavior of road users which are moving straight ahead to theright of the ego-vehicle is relevant for the further planning of thedriving dynamics of the ego-vehicle. If the preceding vehicle were notpresent or did not intend to turn, the behavior of the road users movingstraight ahead would not be relevant for the behavior planning of theego-vehicle.

According to an example embodiment of the present invention, whendriving on the expressway, the simultaneous activation of both turnsignals (hazard lights) of the preceding vehicle may indicate that thisvehicle is approaching the end of a traffic jam or is already situatedat the end of a traffic jam. The ego-vehicle is thus already informedabout having to decelerate soon, or even stop, at a point in time atwhich it is not yet able to measure, or is only able to impreciselydirectly measure, the speed of the preceding vehicle.

In the case of an oncoming vehicle, the activation of the left turnsignal may indicate that this vehicle is preparing to pass anothervehicle driving on the traffic lane driven by it and, for this purpose,will utilize the traffic lane presently driven on by the ego-vehicle. Atthis moment, among others, the speed of the oncoming vehicle, the speedof the vehicle passed by the ego-vehicle, the distance between theego-vehicle and the oncoming vehicle, as well as evasive options at theroadside are relevant for the assessment of whether the situation ispossibly becoming dangerous for the ego-vehicle. As a result of therecognition of the impending passing maneuver already based on the turnsignal, valuable seconds may be gained for the decision whether theego-vehicle, for example, should carry out a braking or evasivemaneuver.

In one particularly advantageous embodiment of the present invention, inthis way, an activation signal for the ego-vehicle is ascertained,taking the assignments, ascertained by the DRNN, of image recordings toclasses representing lamps of the analyzed vehicle into consideration.The ego-vehicle is activated with the aid of this activation signal,which, in turn, may cause an arbitrary suitable intervention in thedriving dynamics of the ego-vehicle. The reliable recognition ofactivated lamps at the vehicle here increases the probability that theintervention in the driving dynamics of the situation detected by theego-vehicle is appropriate.

The present invention also relates to a method for training anencoder-decoder system made up of an ERNN and a DRNN for the use in theabove-described method.

Within the scope of this training method according to the presentinvention, learning image recordings of areas at a vehicle are provided.These learning image recordings are labeled with setpoint assignments toclasses which represent specific lamps of the vehicle. Thus, when alearning image recording is being incorporated into the intermediateproduct in one of the passes of the ERNN, it is desired that the DRNN inthe pass corresponding thereto assigns these learning image recordingsto a class corresponding to the setpoint assignment.

From the learning image recordings, assignments to classes areascertained using the above-described method.

Deviations of the assignments, ascertained with the aid of the DRNN,from the setpoint assignments are assessed based on a predefined costfunction. Parameters which characterize the behavior of the ERNN and ofthe DRNN are optimized, with the goal of the assessment presumablyimproving as a result of the cost function during the further processingof learning image recordings.

As explained above, this training causes the learning image recordingsnot be evaluated in an isolated manner, but in each case in the contextof the remaining learning image recordings at the same vehicle. As aresult, the accuracy of the classification of activated vehicle lamps bythe trained encoder-decoder system is enhanced.

In one particularly advantageous embodiment of the present invention,the setpoint assignment for the learning image recording which wasprocessed in the pass of the ERNN corresponding to the respectivepreceding pass of the DRNN is supplied as input to the second and allsubsequent passes of the DRNN. Thus, for example, the setpointassignment for the first learning image recording which was analyzed inthe preceding first pass of the DRNN is supplied as input to the secondpass of the DRNN, with the aid of which the assignment for the secondlearning image recording is to be ascertained. In this way, it ispresumed during the training that the assignment ascertained in thepreceding pass of the DRNN was correct.

Just the use of the setpoint assignments during further passes of theDRNN causes the success of the training with respect to a certainlearning image recording to become more independent of the pass of theDRNN in which an assignment of this learning image recording to a classis ascertained.

In one further advantageous embodiment of the present invention, theorder in which the learning image recordings are supplied as input tothe various passes of the ERNN is sorted based on a route throughlocations at the vehicle to which these learning image recordingsrelate. In this way, the ability of the encoder-decoder system to justevaluate sequences of inputs, including their inner relationships, isoptimally utilized.

The method may, in particular, be entirely or partiallycomputer-implemented. The present invention thus also relates to acomputer program including machine-readable instructions which, whenthey are executed on one or multiple computer(s), prompt the computer(s)to carry out one of the described methods. Within this meaning, controlunits for vehicles and embedded systems for technical devices, which arealso able to execute machine-readable instructions, are to be regardedas computers.

The present invention also relates to a machine-readable data mediumand/or to a download product including the computer program. A downloadproduct is a digital product transferrable via a data network, i.e.,downloadable by a user of the data network, which may be offered forimmediate download in an online shop, for example.

Furthermore, a computer may be equipped with the computer program, withthe machine-readable data medium and/or with the download product.

Further measures improving the present invention are shown hereafter ingreater detail together with the description of the preferred exemplaryembodiments of the present invention based on figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows one exemplary embodiment of method 100 for recognizingwhich lamps 2 a, 2 b at a vehicle 1 are activated, according to thepresent invention.

FIG. 2 shows one exemplary embodiment of method 200 for training asystem made up of an ERNN 4 and a DRNN 6, according the presentinvention.

FIG. 3 shows one exemplary use of method 100 on an image of a vehicle 1,according to the present invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 shows a schematic flowchart of one exemplary embodiment of method100 for recognizing which lamps 2 a, 2 b at a vehicle 1 are activated.

In step 110, multiple image recordings 3 a through 3 f of candidateareas 1 a through if at vehicle 1 are provided. An activated lamp 2 a, 2b is presumed in each case in candidate areas 1 a through 1 f.

In step 120, image recordings 3 a through 3 f are converted into anintermediate product 5 by executing a recurrent encoder network (ERNN) 4multiple times. In the process, according to block 121, the output of atleast one pass 4 a through 4 f of ERNN 4 is supplied as input to afurther pass 4 a through 4 f of ERNN 4. According to block 122,different image recordings 3 a through 3 f of candidate areas 1 athrough if are supplied as input to different passes 4 a through 4 f ofERNN 4.

In step 130, assignments 7 a through 7 f of image recordings 3 a through3 f of candidate areas 1 a through if to classes are ascertained fromintermediate product 5 by executing a recurrent decoder network (DRNN) 6multiple times. These classes represent specific lamps 2 a, 2 b ofvehicle 1, which is illustrated in FIG. 3 .

In step 140, an activation signal 9 is ascertained for ego-vehicle 8,taking assignments 7 a through 7 f ascertained by DRNN 6 intoconsideration.

In step 150, ego-vehicle 8 is activated with the aid of this activationsignal 9.

According to block 111, candidate areas 1 a through if may be selectedbased on their luminance and/or color from an image or a video of thefront or the rear of vehicle 1. According to block 111 a, imagerecordings 3 a through 3 f may then be condensed into feature vectors z₁through z₆ which indicate the peculiarity of one or multiple feature(s).According to block 123, these feature vectors z₁ through z₆ may then besupplied to ERNN 4 as input, instead of image recordings 3 a through 3f.

According to block 112, candidate areas 1 a through if may be selectedwith the aid of a further neural network from an image or a video of thefront or the rear of vehicle 1. According to block 112 a, this furtherneural network may include at least one convolutional layer, which, byapplying at least one predefined filter kernel to its input, ascertainsa feature map f₁ through f₆ of this input. According to block 124, thisfeature map f₁ through f₆ may then be supplied to ERNN 4 as input,instead of the respective image recording 3 a through 3 f.

According to block 113, image recordings 3 a through 3 f may be recordedwith the aid of at least one sensor which is carried along by anego-vehicle 8.

According to block 125, positions and/or sizes of these image recordings3 a through 3 f relative to vehicle 1 may be supplied to ERNN 4 asinput, in addition to image recordings 3 a through 3 f or featurevectors z₁ through z₆ or feature maps f₁ through f₆ formed thereof.

FIG. 2 shows a schematic flowchart of one exemplary embodiment of method200 for training an encoder-decoder system made up of an ERNN 4 and aDRNN 6 for the use in the above-described method 100.

In step 210, learning image recordings 3 a* through 3 f* of candidateareas 1 a through if at a vehicle 1 are provided. These learning imagerecordings 3 a* through 3 f* are labeled with setpoint assignments 7 a*through 7 f* to classes which represent specific lamps 2 a, 2 b ofvehicle 1. At least one class may also represent the case that therespective learning image recording 3 a* through 3 f* does not include alamp 2 a, 2 b of vehicle 1, but that the light from the respective area1 a through if comes from another source.

In step 220, assignments 7 a through 7 f to classes are ascertained fromlearning image recordings 3 a* through 3 f* using the above-describedmethod 100. In the process, according to block 221, setpoint assignment7 a* through 7 f* for learning image recording 3 a* through 3 f* whichwas processed in pass 4 a through 4 f of ERNN 4 corresponding to therespective preceding pass 6 a through 6 f of DRNN 6 may be supplied asinput to second and all subsequent passes 6 a through 6 f of DRNN 6.

In step 230, deviations of assignments 7 a through 7 f, ascertained withthe aid of DRNN 6, from setpoint assignments 7 a* through 7 f* areassessed based on a predefined cost function 10.

Based on assessment 10 a obtained in the process, in step 240 parameters4*, 6*, which characterize the behavior of ERNN 4 and of DRNN 6, areoptimized, with the goal of assessment 10 a presumably improving as aresult of cost function 10 during the further processing of learningimage recordings 3 a* through 3 f*. The fully optimized states ofparameters 4*, 6* are denoted by reference numerals 4** and 6**.

According to block 222, the order in which learning image recordings 3a* through 3 f* are supplied as input to the various passes 4 a through4 f of ERNN 4 may be sorted based on a route through locations atvehicle 1 to which these learning image recordings relate.

FIG. 3 illustrates one exemplary use of method 100 on an overall imageof a vehicle 1. Based on the color and luminance, six partial images ofcandidate areas 1 a through if at vehicle 1 are cut out of this overallimage as image recordings 3 a through 3 f. According to block 111 a ofmethod 100, these image recordings 3 a through 3 f are condensed intofeature vectors z₁ through z₆.

Feature vectors z₁ through z₆ are supplied to various passes 4 a through4 f of recurrent encoder network (ERNN) 4. These passes 4 a through 4 fare shown separately from one another for better illustration. However,physically, always the same neural network is used. In the example shownin FIG. 3 , the network is a gated recurrent unit (GRU). First pass 4 aonly receives feature vector z₁ as input. Every further pass 4 b through4 f in each case receives the result of the preceding pass 4 a-4 e aswell as a new feature vector z₂ through z₆ as input. The output of thelast pass 4 f is intermediate product 5.

Intermediate product 5 is subsequently analyzed in various passes 6 athrough 6 f of recurrent decoder network (DRNN) 6. These passes 6 athrough 6 f are shown separately from one another for betterillustration. However, physically, always the same neural network isused. In the example shown in FIG. 3 , the network is a gated recurrentunit (GRU). First pass 6 a receives intermediate product 5 as input. Allfurther passes 6 b through 6 f receive the output of the respectivepreceding pass 6 a through 6 e as input. Each pass 6 a-6 f supplies, asoutput, an assignment 7 a through 7 f of feature vector z₁ through z₆,processed in the corresponding pass 4 a through 4 f of ERNN 4, to one ormultiple class(es).

In the example shown in FIG. 3 , assignment 7 a indicates that imagerecording 3 a shows a rear light and a turn signal as lamp 2 a ofvehicle 1. Assignments 7 b through 7 e show that the respective imagerecordings 3 b through 3 e do not show a lamp 2 a, 2 b of vehicle 1, butinstead in each case light from another source which is reflected byvehicle 1. This is symbolized in each case by a cross (x). Assignment 7f indicates that image recording 3 f shows a rear light as lamp 2 b ofvehicle 1.

What is claimed is:
 1. A method for recognizing which lamps at a vehicleare activated, including the steps: providing multiple image recordingsof candidate areas at the vehicle in which an activated lamp ispresumed; converting the image recordings into an intermediate productby executing a recurrent encoder network (ERNN) multiple times, theoutput of at least one pass of the ERNN being supplied as input to afurther pass of the ERNN, and different image recordings of candidateareas being supplied as input to different passes of the ERNN; andascertaining assignments of the image recordings of candidate areas toclasses which represent specific lamps of the vehicle from theintermediate product by executing a recurrent decoder network (DRNN)multiple times, the output of at least one pass of the DRNN is suppliedas input to a further pass of the DRNN, and in each pass of the DRNN, anassignment of the image recording which was processed in the pass of theERNN corresponding to the pass of the DRNN is ascertained to at leastone class.
 2. The method as recited in claim 1, wherein the candidateareas are selected based on their luminance and/or color from an imageor a video of the front or the rear of vehicle.
 3. The method as recitedin claim 2, wherein the image recordings are condensed into featurevectors, which indicate a peculiarity of one or multiple features, thefeature vectors being supplied as input to the ERNN instead of the imagerecordings.
 4. The method as recited in claim 1, wherein the candidateareas are selected using a further neural network from an image or avideo of the front or the rear of vehicle.
 5. The method as recited inclaim 4, wherein the further neural network includes at least oneconvolutional layer, which, by applying at least one predefined filterkernel to its input, ascertains a feature map of the input, the featuremap being supplied as input to the ERNN instead of the image recordings.6. The method as recited in claim 1, wherein positions and/or sizes ofthe image recordings relative to the vehicle are also supplied to theERNN as inputs.
 7. The method as recited in claim 1, wherein the ERNNand/or the DRNN have no more than 100 neurons.
 8. The method as recitedin claim 1, wherein the image recordings are recorded using at least onesensor which is carried along by an ego-vehicle.
 9. The method asrecited in claim 8, wherein: an activation signal for the ego-vehicle isascertained, taking the assignments ascertained by the DRNN intoconsideration, and the ego-vehicle is activated using the activationsignal.
 10. A method for training an encoder-decoder system made up ofan recurrent encoder network (ERNN) and a recurrent decoder network(DRNN), comprising the following steps: providing learning imagerecordings of areas at a vehicle, the learning image recordings beinglabeled with setpoint assignments to classes which represent specificlamps of the vehicle; ascertaining assignments to classes from thelearning image recordings by: providing the image recordings ofcandidate areas at the vehicle in which an activated lamp is presumed;converting the image recordings into an intermediate product byexecuting the ERNN multiple times, the output of at least one pass ofthe ERNN being supplied as input to a further pass of the ERNN, anddifferent image recordings of candidate areas being supplied as input todifferent passes of the ERNN, ascertaining the assignments of the imagerecordings of candidate areas to the classes which represent specificlamps of the vehicle from the intermediate product by executing the DRNNmultiple times, the output of at least one pass of the DRNN is suppliedas input to a further pass of the DRNN, and in each pass of the DRNN, anassignment of the image recording which was processed in the pass of theERNN corresponding to the pass of the DRNN is ascertained to at leastone class; assessing deviations of the assignments, ascertained usingthe DRNN, from the setpoint assignments, based on a predefined costfunction; and optimizing parameters which characterize a behavior of theERNN and of the DRNN, with a goal of the assessment improving as aresult of the cost function during the further processing of learningimage recordings.
 11. The method as recited in claim 10, wherein thesetpoint assignment for each learning image recording which wasprocessed in the pass of the ERNN corresponding to the respectivepreceding pass of the DRNN is supplied as input to the second and allsubsequent passes of the DRNN.
 12. The method as recited in claim 10,wherein an order in which the learning image recordings are supplied asinput to the various passes of the ERNN is sorted based on a routethrough locations at the vehicle to which the learning image recordingsrelate.
 13. A machine-readable data medium on which is stored a computerprogram including machine-readable instructions for recognizing whichlamps at a vehicle are activated, the instructions, when executed by acomputer, causing the computer to perform the following steps: providingmultiple image recordings of candidate areas at the vehicle in which anactivated lamp is presumed; converting the image recordings into anintermediate product by executing a recurrent encoder network (ERNN)multiple times, the output of at least one pass of the ERNN beingsupplied as input to a further pass of the ERNN, and different imagerecordings of candidate areas being supplied as input to differentpasses of the ERNN; and ascertaining assignments of the image recordingsof candidate areas to classes which represent specific lamps of thevehicle from the intermediate product by executing a recurrent decodernetwork (DRNN) multiple times, the output of at least one pass of theDRNN is supplied as input to a further pass of the DRNN, and in eachpass of the DRNN, an assignment of the image recording which wasprocessed in the pass of the ERNN corresponding to the pass of the DRNNis ascertained to at least one class
 14. One or multiple computersconfigured to recognize which lamps at a vehicle are activated, the oneor multiple computers configured to: provide multiple image recordingsof candidate areas at the vehicle in which an activated lamp ispresumed; convert the image recordings into an intermediate product byexecuting a recurrent encoder network (ERNN) multiple times, the outputof at least one pass of the ERNN being supplied as input to a furtherpass of the ERNN, and different image recordings of candidate areasbeing supplied as input to different passes of the ERNN; and ascertainassignments of the image recordings of candidate areas to classes whichrepresent specific lamps of the vehicle from the intermediate product byexecuting a recurrent decoder network (DRNN) multiple times, the outputof at least one pass of the DRNN is supplied as input to a further passof the DRNN, and in each pass of the DRNN, an assignment of the imagerecording which was processed in the pass of the ERNN corresponding tothe pass of the DRNN is ascertained to at least one class.